Multimodal corpora for human-machine interaction research
نویسندگان
چکیده
In recent years human-machine interaction has increased its importance. One approach to an ideal human-machine interaction is develop a multi-modal system behaves like human-beings. This paper introduces an overview on multimodal corpora which are currently developed in Japan for the purpose. The paper describes database of 1)Multi-modal interaction, 2)Audio-visual speech, 3)Spoken dialogue with multiple speakers, 4)Gesture of sign language and 5)Sound scene data in real acoustic environments.
منابع مشابه
The SmartWeb Corpora: Multimodal Access to the Web in Natural Environments
As a result from the German SmartWeb project three speech corpora, one of them multimodal, have been published by the Bavarian Archive for Speech Signals (BAS). They contain speech and video signals from human–machine interactions in real indoor and outdoor environments. The scenarios for these corpora are a typicial handheld PDA interaction (SHC), an interaction on a running motorcycle (SMC) a...
متن کاملMultimodal Comparable Corpora as Resources for Extracting Parallel Data: Parallel Phrases Extraction
Discovering parallel data in comparable corpora is a promising approach for overcoming the lack of parallel texts in statistical machine translation and other NLP applications. In this paper we propose an alternative to comparable corpora of texts as resources for extracting parallel data: a multimodal comparable corpus of audio and texts. We present a novel method to detect parallel phrases fr...
متن کاملLEarning and TEaching corpora (LETEC): data-sharing and repository for research on multimodal interactions
The number of online environments language teachers can employ is constantly growing, offering increased potential for multimodal L2 interaction analysis. This paper introduces the LEarning and TEaching Corpora (LETEC) methodology that links, following international standards, all elements resulting from an online learning situation, whose context is described by a pedagogical scenario and a re...
متن کاملFrom Annotated Multimodal Corpora to Simulated Human-Like Behaviors
Multimodal corpora prove useful at different stages of the development process of embodied conversational agents. Insights into human-human communicative behaviors can be drawn from such corpora. Rules for planning and generating such behavior in agents can be derived from this information. And even the evaluation of human-agent interactions can rely on corpus data from human-human communicatio...
متن کاملParallel Texts Extraction from Multimodal Comparable Corpora
Statistical machine translation (SMT) systems depend on the availability of domain-specific bilingual parallel text. However parallel corpora are a limited resource and they are often not available for some domains or language pairs. We analyze the feasibility of extracting parallel sentences from multimodal comparable corpora. This work extends the use of comparable corpora by using audio sour...
متن کامل